Target tracks produced by EO/IR sensors often need to be projected onto the surface of the Earth for subsequent display or for fusion with tracks produced by other sensors. A bias in the parameters used for this projection, for example, heading or pitch of the platform that carries the sensors, leads to a projective bias in the ground tracks. When target tracks produced by a video tracker are projected to the Earth's surface, they often become a rotated, stretched and translated version of the true tracks due to sensor calibration errors. If the correspondence between the projected tracks and the roads they are on can be established, a homography can then be computed and the projection bias can be removed. This correspondence is typically easy to establish by a human operator. However, it is desirable to have an automated solution to reduce an operator's work load, and the problem is challenging due to the lack of fiduciary targets. A fiduciary target is a point on a track whose position on a corresponding road is known. These fiduciary targets are not automatically generated in surveillance settings and such fiduciary targets to correct for bias errors are therefore unavailable for use in bias correction.
More particularly, there is a need to align the tracks projected from an overflying video camera onto road networks. The tracks referred to herein correspond to positions of a moving target. As the target moves along it is detected by different sensors, such as, for instance those associated with radar measurements, and for instance, video cameras mounted on highflying unmanned aerial vehicles. These sensors look down and identify cars or vehicles. The purpose of such a system is to be able to track the vehicles on a map and to identify these vehicles at projections on corresponding roads.
Moreover there is a further need for fusing tracking information from different modalities and different sensors, and to do so in one coherent picture. For instance, if the tracks are those obtained from radars, the projected radar tracks are to be aligned with corresponding tracks obtained from video cameras. It is, therefore, important that these projected tracks be fused. As a result, when the vehicles are detected by multiple sensors it is important to be able to align the projected tracks of each of these sensors with themselves as well as with the road network.
With the problem of bias, if the camera is mounted on a stable platform at a certain altitude, there may be a slight deviation of the platform from the stable position. But that slight deviation can rotate a projected track off a road. Thus, these slight deviation-caused bias errors need to be canceled out so that the tracks projected from an aerial reconnaissance vehicle align with the road network.
By way of further background, typically when a vehicle is tracked utilizing a video framework it is common to put a box around a vehicle and then follow the vehicle. The box can completely surround that particular vehicle, and when it is determined where the vehicle is positioned, the location of the vehicle must then be projected on the ground. Because of bias errors, the vehicle may be, for example, 5 meters off road in any given direction. Knowing the projected position of the vehicle on the ground, one then needs to define a translation in terms of a contraction or elongation corresponding to the bias so as to be able to move the projected position of the vehicle to the true or actual position on the road and correct the bias.
Moreover, in the past when video tracks were taken and projected on the ground, and subsequently filled with radar tracks, these tracks did not overlie each other. This error is because the two tracks did not align with the road and did not align with themselves. The correspondence between the video and radar tracks, and between these tracks and a true position on the ground, can be established in a manual process and then tabulated to rotate the tracks back to the road. However, as mentioned herein, this process is not a satisfactory solution since it requires manual bias correction.
Track bias removal has been studied extensively and many methods have been proposed in the prior art. One study includes considerations of the problem of removing biases by the fusion center from state estimates produced by local, bias-ignorant trackers, when track-to-track association is known. Another study analyzes the problem of calculating the probability of track-to-track association given data from biased sensors. Another study has analyzed the problem of jointly obtaining the optimal track-to-track association and the estimation of the (relative, additive) sensor bias is formulated as a nonlinear mixed integer programming problem and solved using a multistart local search heuristic.
Still further studies have obtained matchings of biased tracks to roads by treating both the road network and the tracks as binary images, and using feature finding methods in image processing. When roads are dense and similar, the most prominent features are often the turns or intersections. This study considered not only the turns, but all points on a track/road. In a similar study, measurements were filtered into tracks with road map assistance, and intersecting roads were handled with an interacting multiple model (IMM) scheme. More recent studies have dealt with bias estimation/removal through the fusion of possible biased local estimates in sensor network based on sensor selection, through estimation of optical sensor measurements with targets of opportunity, and with bias estimation for practical distributed multiradar-multitarget tracking systems, among others.
Some prior art have used Global Positioning Systems (GPS) to propose solutions. However, a problem with GPS is that in order to obtain a smooth track on the road, based on recorded noisy dots that can be off the road, the system has to account for several sources of noises. As a result, the criteria on the quality of the obtained tracks have to take many factors into account. Thus, solutions to the GPS problem do not readily provide a solution to the problem of projection bias removal, which is narrowly but precisely formulated and requires a precise solution.
All of the proposed solutions to the topic have shortcomings which make these solutions inadequate for correcting the biased track positions to their true track positions. Thus, a heretofore unaddressed need exists in the industry to address the aforementioned deficiencies and inadequacies.